The invention relates to a head-mounted display (HMD) for determining the posture of the head-mounted display in a vehicle.
Nowadays, head-mounted displays (HMDs) are known which make it possible to display information to the wearer of the head-mounted display are known. The head-mounted display is here worn on the head like a normal pair of eyeglasses (spectacles) used as a vision aid. However, unlike a normal pair of spectacles, the head-mounted display includes a display which is arranged near the eye or eyes of the user when the head-mounted display is worn. The display can have two partial displays, one for each eye. Information in the form of text, graphical representations or mixtures thereof can be displayed to the user using the display. The display can, in particular, be partially transmissive, that is to say it can be configured such that the wearer can also see the environment behind the display. With particular preference, the information is displayed to the wearer in a contact analog fashion, which is sometimes also referred to as augmented reality. Here, the information is displayed to the wearer of the head-mounted display at a location which is oriented toward the location of an object in the environment, that is to say for example adjoining the object or overlaying the object. In order to implement the contact analog manner, typically the position of the object in the environment and the posture of the head-mounted display relative to the object must be known, that is to say the position of the head-mounted display and the alignment of the head-mounted display.
Head-mounted displays which likewise include in each case a camera which takes recordings in the viewing direction of the wearer of the head-mounted display have become known, see for example WO 2013/012914.
Head-mounted displays can also be used in vehicles to display contact analog information. Such information can relate to other traffic participants or objects within the vehicle. In order to be able to display contact analog information, in most cases the posture of the head-mounted display must also be known.
It is an object of the present invention to provide a method with which the posture of the head-mounted display can be determined and, based thereon, information can be displayed in contact analog fashion.
One aspect of the invention relates to a head-mounted display which is configured for determining the posture of the head-mounted display relative to a vehicle, wherein the head-mounted display includes a display and a camera, as well as an electronic computation device (e.g. a microprocessor). The apparatus is configured and executes processing for: taking recordings of the environment of the head-mounted display using the camera; receiving statements relating to the position and/or alignment of an object which is movable (relative to the vehicle's interior) of the interior of the vehicle (for example, wirelessly using Wi-Fi or Bluetooth standards); detecting the image of the object in the recordings of the camera; determining the posture of the head-mounted display by taking into consideration the image of the object in the recordings relative to the vehicle. The posture of the head-mounted display indicates the 3-D position of the head-mounted display and the 3-D alignment of the head-mounted display relative to the vehicle. Statements relating to the position of the object can be statements relating to the position of features of the object which are suitable for detecting and determining the position and posture of the head-mounted display. One object can be, for example, a steering wheel, a door or a lever.
What is proposed here in other words is the determination of the posture of the head-mounted display using movable objects of the vehicle, in particular a steering wheel, wherein the position and/or alignment of the object are known. Since the rotation (steering angle) and position (height and length setting) of a steering wheel change during operation of the vehicle, the steering wheel cannot, in accordance with the prior art, serve for determining the posture of the head-mounted display using camera recordings of the steering wheel. The invention provides that the steering angle and the position of the steering wheel are transmitted from the vehicle to the head-mounted display (for example, wirelessly using the Wi-Fi or Bluetooth standards). With this knowledge, the recordings of the steering wheel can be used to infer the posture of the head-mounted display. It may be possible for posture determination to also use the recordings of other stationary objects in the interior at the same time.
Using the received statements, the head-mounted display can establish a 3-D model of the movable object and use it for posture determination. The 3-D model of the object can be derived from a generic 3-D model on the basis of the statements relating to position and alignment of the object. Moreover, the head-mounted display can be provided with a 3-D model also of other parts of the interior of the vehicle.
In one implementation, the apparatus is furthermore configured and executes processing for: receiving statements relating to the change rate of the position and/or alignment of the object of the interior of the vehicle. Using these statements, the head-mounted display can determine the probable future position and alignment of the object. This is helpful primarily if there is a time period between the measurement time of the position and the alignment of the object and the determination of the posture of the head-mounted display of which it can be assumed that the position and alignment of the object change sufficiently strongly to significantly influence the posture determination.
The detection of the image of the object is typically carried out in dependence on the received statements relating to the position and/or alignment of the object. The detection can be carried out in this way more robustly and faster because of the previous knowledge with respect to the alignment and position of the object. The detection can be carried out using pattern recognition or a classifier. Here, the pattern recognition or the classifier is adapted to the statements relating to the position and/or alignment of the object.
The position of the steering wheel is typically obtained from the height and length setting of the steering wheel by the driver. The alignment of the steering wheel or its rotation is obtained from the steering angle, that is to say the rotation of the steering wheel about the steering axis.
In one implementation, the determination of the posture of the head-mounted display is carried out even with taking into consideration non-movable objects of the vehicle's interior. The posture determination is thus carried out based on a mix of static and dynamic features which are detected in each case on objects which are movable in the interior and objects which are static. A static object can be, for example, an instrument cluster, a display, a trim panel, a window frame, etc. These objects can be represented in a 3-D CAD model and serve for posture determination.
The 3-D CAD model can furthermore serve for identifying which regions of static objects can be concealed by movable objects. In this way, features of static objects can be used for posture determination even if the static objects are at least partially covered by movable objects. Here, the current (received) position and alignment of the respective movable object can be taken into consideration.
In one advantageous implementation, the apparatus is furthermore configured and executes processing for: checking whether a number of objects and/or features of objects which is sufficient for determining the posture is detectable in the recordings of the camera; if no sufficient number is detectable: carrying out the steps of receiving, detecting and determining. The above-illustrated methods can be used as alternative methods, if no sufficient number of static features (that is to say features which have been detected as spatially fixed) of the vehicle's interior are detected using the camera. Dynamic features are also taken into consideration (that is to say the detection of features of movable objects) only if the posture of the head-mounted display cannot be determined using the static features.
Another aspect of the invention relates to a method for determining the posture of a head-mounted display in a vehicle, wherein the head-mounted display includes a display, a camera and an electronic computation device; wherein the method comprises: taking recordings of the environment of the head-mounted display using the camera; receiving statements relating to the position and/or alignment of an object of the interior of the vehicle; detecting the image of the object in the recordings of the camera; determining the posture of the head-mounted display by taking into consideration the image of the object in the recordings; wherein the posture of the head-mounted display indicates the position of the head-mounted display and the alignment of the head-mounted display. The method is typically carried out by the head-mounted display.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.